
Amendments to the Claims 

This listing of claims will replace all prior versions, and listings, of the claims: 

1. - 16. (canceled) 

17. (new) A method for determining accuracy of a classifier that classified an item in a 
hierarchy of categories, the method comprising: 

assigning a first indication to each category in the hierarchy to which the item 
actually belongs; 

assigning a second indication to each category in the hierarchy to which the item 
actually does not belong; 

assigning a third indication to each category in the hierarchy that the classifier 
selected for the item; 

assigning a fourth indication to each category in the hierarchy that the classifier 
did not select for the item; and 

evaluating the first, second, third, and fourth indications to determine an accuracy 
of the classifier that classified the item in the hierarchy of categories. 

18. (new) The method of claim 17 wherein the accuracy provides an indication of a 
degree of correctness rendered by the classifier in classifying the item. 

19. (new) The method of claim 1 7 wherein the accuracy provides a measure between a 
true classification of the item in the hierarchy and classification of the item rendered by 
the classifier. 

20. (new) The method of claim 17 further comprising: 

providing a numerical range to indicate accuracy of the classifier; 
combining the first, second, third, and fourth indications to determine a numerical 
value within the numerical range to indicate the accuracy of the classifier. 
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21. (new) The method of claim 17 further comprising: 

assigning the first indication to all ancestors of a category that are assigned a first 
indication. 

22. (new) The method of claim 17 wherein evaluating the first, second, third, and fourth 
indications further comprises counting the first, second, third, and fourth indications. 

23. (new) The method of claim 17 further comprising: 

assigning an indifferent indication to each category in the hierarchy that does not 
contribute to a measure of accuracy of the classifier. 

24. (new) The method of claim 17 wherein evaluating the first, second, third, and fourth 
indications further comprises: 

determining a first tendency of the classifier to put the item in more categories 
than the item actually belongs; 

determining a second tendency of the classifier to put the item in less categories 
than the item actually belongs. 

25 . (new) The method of claim 24 further comprising: 

averaging the first and second tendencies to obtain the accuracy of the classifier. 

26. (new) The method of claim 24 further comprising: 

combining the first and second tendencies with a harmonic mean to obtain the 
accuracy of the classifier. 

27. (new) A method, comprising: 

assigning a first indication to each category in a hierarchy to which an item 
actually belongs; 

assigning a second indication to each category in the hierarchy to which the item 
actually does not belong; 

classifying, with a classifier, the item in categories in the hierarchy; 
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assigning a third indication to each category in the hierarchy that the classifier 
selected for the item; 

assigning a fourth indication to each category in the hierarchy that the classifier 
did not select for the item; and 

combining the first, second, third, and fourth indications to determine a degree of 
correctness rendered by the classifier in classifying the item in the hierarchy. 

28. (new) The method of claim 27 further comprising: 

assigning the first indication as being true; 
assigning the second indication as being false; 
assigning the third indication as being positive; 
assigning the fourth indication as being negative. 

29. (new) The method of claim 27 further comprising: 

assigning the first indication to each category in the hierarchy to which a second 
item actually belongs; 

assigning the second indication to each category in the hierarchy to which the 
second item actually does not belong; 

classifying, with the classifier, the second item in categories in the hierarchy; 

assigning the third indication to each category in the hierarchy that the classifier 
selected for the second item; 

assigning the fourth indication to each category in the hierarchy that the classifier 
did not select for the second item; and 

combining the first, second, third, and fourth indications for the item with the 
first, second, third, and fourth indications for the second item to determine a measure of 
accuracy of the classifier in classifying items in the hierarchy. 

30. (new) A method, comprising: 

assigning a true indication to at least a first category in a hierarchy to which an 
item actually belongs; 
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assigning a false indication to at least a second category in the hierarchy to which 
the item actually does not belong; 

classifying, with a classifier, the item in the first category to obtain a first 
indication; 

classifying, with the classifier, the item in the second category to obtain a second 
indication; and 

comparing the true and false indications with the first and second indications to 
determine a degree of correctness rendered by the classifier in classifying the item in the 
hierarchy. 
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